Propagate block hashes instead of full blocks (network breaking) #1986
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sdbondi
force-pushed
the
sb-core-block-hash-proto
branch
2 times, most recently
from
7d223ce to
de31f7e
Compare
CjS77
previously approved these changes
Jun 14, 2020
- Nodes propagate a block hash (`NewBlock` message = 33 bytes) instead of the full block. This will significantly reduce network bandwidth usage for block propagation. - On receipt of a block hash a node checks if it has the block. If so, it simply ignores the message. Otherwise, it requests the full block from the peer that sent the `NewBlock` message. If the block is valid and has been added to the node's blockchain db, the node propagates the block hash message to other peers. - Change block propagate test to tests a few block hashes being propagated rather than a single block. - Changed invalid block test to check that an invalid block is not proagated. - New test to check that an invalid block hash is not propagated - Cleaned up some unecessary generics on structs
sdbondi
force-pushed
the
sb-core-block-hash-proto
branch
2 times, most recently
from
0f5e66d to
4159dc3
Compare
sdbondi
added a commit
to sdbondi/tari
that referenced
this pull request
Sep 16, 2020
This PR changes the `Flood` broadcast stragey to "flood" a message to every _connected_ peer node. Since PR tari-project#1986, block hashes are propagated across the network and the full block only downloaded once per peer (push became push-pull gossip). This large drop in propagation overhead means that we can afford to spread the hash a little more widely and use the modified `Flood` stragegy. The main change is to change the flood strategy from sending to _all_ valid peers in the peer list, to only sending to currenty connected peers (inbound and outbound connections). As before, client nodes are exempt from flood propagation messages.
sdbondi
added a commit
to sdbondi/tari
that referenced
this pull request
Sep 16, 2020
This PR changes the `Flood` broadcast stragey to "flood" a message to every _connected_ peer node. Since PR tari-project#1986, block hashes are propagated across the network and the full block only downloaded once per peer (push became push-pull gossip). This large drop in propagation overhead means that we can afford to spread the hash a little more widely and use the modified `Flood` stragegy. The main change is to change the flood strategy from sending to _all_ valid peers in the peer list, to only sending to currenty connected peers (inbound and outbound connections). As before, client nodes are exempt from flood propagation messages.
12 tasks
sdbondi
added a commit
to sdbondi/tari
that referenced
this pull request
Sep 16, 2020
This PR changes the Flood broadcast stragey to "flood" a message to every connected peer node. Since PR tari-project#1986, block hashes are propagated across the network and the full block only downloaded once per peer (push became push-pull gossip). This large drop in propagation overhead means that we can afford to spread the hash a little more widely and use the modified Flood strategy. Previously, the hash propagation used the propagate strategy which only sends to 4 random connected peers. The flood strategy is changed from sending all valid peers in the peer list, to only sending to currently connected peers (inbound and outbound connections). As before, client nodes are exempt from flood propagation messages.
stringhandler
added a commit
that referenced
this pull request
Sep 22, 2020
This PR changes the (previously unused) Flood broadcast strategy to send a message to every connected peer node. Since PR #1986, block hashes are propagated across the network and the full block is only downloaded once per peer (push became push-pull gossip). This large drop in propagation overhead means that we can afford to spread the hash a little more widely and use the modified Flood strategy. Previously, the hash propagation used the propagate strategy which only sends to 4 random connected peers.
StriderDM
pushed a commit
to StriderDM/tari
that referenced
this pull request
Oct 2, 2020
This PR changes the Flood broadcast stragey to "flood" a message to every connected peer node. Since PR tari-project#1986, block hashes are propagated across the network and the full block only downloaded once per peer (push became push-pull gossip). This large drop in propagation overhead means that we can afford to spread the hash a little more widely and use the modified Flood strategy. Previously, the hash propagation used the propagate strategy which only sends to 4 random connected peers. The flood strategy is changed from sending all valid peers in the peer list, to only sending to currently connected peers (inbound and outbound connections). As before, client nodes are exempt from flood propagation messages.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
NewBlockmessage = 33 bytes) insteadof the full block. This will significantly reduce network bandwidth
usage for block propagation.
If so, it simply ignores the message. Otherwise, it requests the
full block from the peer that sent the
NewBlockmessage. If theblock is valid and has been added to the node's blockchain db,
the node propagates the block hash message to other peers.
than a single block.
propagated.
Motivation and Context
Significantly reduce the bandwidth required for block propagation.
The benefits are twofold:
Possible issues:
InboundNodeCommsHandler::handle_new_block_messagefunction is able to correctly handle duplicates by sequentially querying all nodes that sent the block hash message. However, due to message dedup, the message is only received once. If the full block is not returned from the source node (for whatever reason) then the node will miss the block and have to transition to block sync state to catch up.How Has This Been Tested?
Types of changes
Checklist:
developmentbranch.cargo-fmt --allbefore pushing.